Digital Camera System for Recording, Editing and Visualizing Images

ABSTRACT

A digital camera system ( 20 ), as illustrated in FIG.  1 , includes an optical assembly ( 22 ) to gather light ( 24 ) from a desired scene ( 26 ), a modular imaging subsystem ( 28 ) aligned with the optical assembly ( 22 ), and an image processing, recording and display subsystem ( 34 ).

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority to and the benefit of U.S. ProvisionalApplication Ser. No. 60/923,339, the entire content of which isincorporated herein by reference.

FIELD OF THE INVENTION

The present invention is related, in general, to a digital cinema camerasystem. More particularly, the present invention is related to a digitalcinema camera system for recording, editing and visualizing images.

BACKGROUND OF THE INVENTION

For many years, film cameras were the only option for capturing cinemaquality motion pictures. The time requirements and costs related toshooting and processing motion picture images on film stock and thentransferring those images into a digital form have created a need formotion picture cameras that capture high definition or cinema resolutionimagery directly in a digital form. The advent of Digital Cinemas, costeffective Stereo 3D Digital cinema projection systems and establishmentof Digital Cinema Initiative SMPTE Standards has fueled the need formore content creation for delivery at 2K, 4K and Stereo formats.

Accordingly, there is a need for a digital camera system that meets theneeds described above that reduces costs. There is also a need for adigital camera system that leverages digital processing andvisualization tools. There is a further need for a digital camera systemthat provides user feedback and metadata collection when shootingspecial effects, compositions and stereo or multi-camera content. Thereis an additional need for a digital camera system that improvesflexibility in networked collaboration, enables separated imaging blockand recording, has a simple workflow with metadata management and,importantly, maintains film-like reproduction qualities and cameraoperation. There is also a need for a digital camera system that notonly provides the capabilities described above but can also utilizeexisting cabling infrastructure, broadcast signal monitoring andtransmission systems. There is a need for a digital camera system thatmixes broadcast standard digital sources into a recording andvisualization system, as well as generate broadcast standard and networkstreaming outputs for 2D and 3D content.

In the past few years, while several digital cinema cameras have emergedon the market these digital cinema cameras are complex designs withlimited connectivity that are only able to address a limited set of theneeds described above. For example, these digital cinema cameras areincompatible with existing cable infrastructure. Also, these digitalcinema cameras either completely lack network management or are capableof only minimal network management (i.e., only simple controls that lackfull image streaming or metadata management). Further, these digitalcinema cameras lack the ability to capture or record multi-sensor 2K or4K image data using a single control application. Additionally, thesedigital cinema cameras lack visualization tools or metadata integration.These digital cinema cameras do not utilize existing broadcastinfrastructure to transmit multi-resolution data and have complexworkflows with respect to stereo 3D and multi-camera contentacquisition, broadcast and network transmission either live or in a postproduction process. These digital cinema cameras are limited to1920×1080 image sensor pixel arrays that require the use of a multiplesensor prism block which, in turn, requires use of complex and expensiveoptics. These digital cinema cameras utilize dedicated hardwarefunctions with no or limited image processing flexibility or upgradecapability. Dedicated hardware functions utilized by these digitalcinema cameras include video processing to perform non-reversible colorspace transformations or sub-sampling to formats, such as YUV 4:2:2 and4:4:4, as standard broadcast signals. These digital cinema camerasimplement a variety of proprietary compression and coding schemes thatintroduce visible image artifacts, especially when projected on largescreens. While a number of these digital cinema cameras can generatepreview imagery for display on an electronic viewfinder, these digitalcinema cameras can only do so with limited resolution or visualizationtools. High-resolution outputs from these digital cinema cameras arerestricted to transmission in SMPTE standard resolution and formats.These digital cinema cameras often output imagery to be record onrestrictive, proprietary or large external storage devices. Thesestorage devices include a tape storage system having only linear dataaccess, Non-Volatile Flash or RAM drives with limited storage, andmultiple hard disk drive RAID storage systems which are oftennon-portable and whose media cannot be easily removed or transported fordirect use in other systems. Also, the files stored on these storagedevices have limited color correction, image processing or postproduction metadata integration.

In recent years, many digital still cameras or dual-mode video and stillcamcorders have also been developed which use single image sensors withcolor filter arrays. These digital still cameras and camcorder devicesdo use higher resolution sensors (e.g., HD (1920×1080) camcorders,digital single-lens reflex camera (DSLR) are now 10 MP and higher).However, these digital still cameras and camcorders have slow readoutarchitectures (e.g., a DSLR may only shoot four (4) frames per second)and can only achieve video rate preview at low resolution (e.g.,640×480) or standard definition (e.g., VGA 640×480 at thirty (30) framesper second) using sub-sampling or windowing techniques. These digitalstill cameras and camcorders use dedicated hardware functions ortargeted function digital signal processors (DSP) to perform imageprocessing to interpolate and colorize the raw image data from the imagesensor. These digital still cameras and camcorders compress thecolorized images for storage; but the compressing process performed bythese devices prevents access to the original full raw image pixel datafor later processing, analysis or colorization. In addition, theinterpolation and color processing applied to the source raw data inthose devices initially generates data sets that are larger than thesource raw data which, in turn, requires the application of highercompression to fit the data sets into a target storage capacity. Thistypically results in a reduction in image quality compared to theoriginal image or a coded version of the raw data.

A few single sensor cameras have been developed for use in 2K and 4Kacquisitions in raw format. However, these cameras use dedicatedhardware or targeted function DSPs to perform image processing tointerpolate, colorize and display preview quality output andsimultaneously compress the raw sensor image data for later digitalediting and grading. Also, the compression method and metadata employedby these cameras foreclose the dynamic retrieval of alternativeresolution or representations at different bit rates during recordingfor network streaming, remote grading or adaptive editing. Due to theirarchitectures, these single sensor cameras must apply high compressionto fit data into target internal storage capacity devices. Also, due totheir architectures, these single sensor cameras lack the ability totransmit the imager raw content over existing broadcast or networkinfrastructure cabling for remote monitoring, networking, recording orvisualization. These single sensor cameras cannot process capturedsignals with prerecorded content or broadcast format signals for livecomposition, switching, grading, mixing into virtual sets or addinggraphic overlays based on extracted metadata or analytics. These singlesensor cameras also lack the ability to manage, control or recordmulti-sensor imagers, which may be remotely connected to a recordingsystem.

In recent years, there has been an interest in producing digital cinemaquality 3D stereographic, wide-dynamic and immersive content usingmultiple imagers. This has created a need for more efficient modular andscalable cameras and workflow solutions. There is a further need for adigital camera system having a precise synchronization mechanism toenable images to be mixed or stitched without motion artifacts. Whiledigital camera systems have been used to produce this type of content,these camera systems suffer from the same system limitations as thecameras described above. These camera systems are mostly comprised ofstand-alone cameras, each with individual controls, viewing andrecording systems, with no integration mechanism other than a commonsync signal (i.e., there is no communication between camera controls orviewing and recording settings). These camera systems are large andbulky such that the camera systems cannot be placed very close togetherphysically, as is required for short inter-ocular distances in 3Dstereographic or for creating hemispherical views where cameras need tobe placed as close together as possible from a common center point. Whenshooting thru mirrors and beam splitters, rigs (i.e., a combination ofdigital cameras, optics and mounting platform) become more cumbersomeand difficult to use in handheld shooting environments. Finally, thesecamera systems lack a comprehensive set of image processing,visualization, positioning control, recording, playback, communicationsand display tools for use in such high-definition multi-camera systems.

SUMMARY OF THE INVENTION

The present invention as described herein discloses a digital camerasystem that captures scalable resolution, bit-depth and frame rate rawor color processed images from one or more modular imaging modules atprecise film or video rates, can utilize industry standard cablinginfrastructure for transmitting either the raw sensor data or processedraw on the same or different links, provides a mechanism for timingsynchronization of exposure and readout cycles from multiple imagingmodules, uses a unified software or operator interface to control thecapture, processing and non-destructive visualization from one or moreimaging modules, can optionally combine the live imagery with previouslystored imagery or computer generated virtual sets and simultaneouslyrecord the raw, broadcast format, or visualization processed imagery inits original or near original representation. The processor can be usedto compress the imagery with an encoder, which can generate multiplestreams one for the purpose of recording at highest quality andoptionally additional streams at lower data rates for remotetransmission. It enables the recording of one or multiple image streamsusing a common removable storage device or across multiple devices forincreased throughput. The recording can make use of a single filecontaining the streams from multiple imaging modules with metadata toenable the selective playback of one or more streams. The outputprocessing can include mixing the imagery from the multiple streams fordisplay on standard computer or broadcast monitoring devices orprocessed for output on specialized stereographic displays that requireformatting and synchronization from dual image streams. Utilizingmetadata encoded in the recorded stream or generated thru user inputduring playback the relative position, color transformation and formatof the dual streams, representing the left and right eye content, can beadjusted to change the stereographic effect and depth perception onthese displays.

This invention enables reduced complexity for capturing imagery from oneor more image modules, enables remote image sensing and frame grabbingwith transmission using existing industry standard broadcast andnetworking infrastructure, improves storage and processing efficiency,provides increased flexibility and tools visualization, networking,analysis and mixing of prerecorded or computer generated data, anddelivers unique display modes 2D and 3D representation of the multiplestreams during live, recording, playback or post processing. Thedisclosed digital camera system may include optics, one or more imagingmodules, a frame grabber, a processor, software, user input mechanism, adisplay, synchronization mechanism, networking means and storage means.In addition, a configuration is disclosed for a portable digital cameraand recording system capable of HD, 2K and 4K stereo-3D or wide-dynamicmulti-image acquisition using two image sensing modules and separatedimage processing, recording and display subsystem.

Other features and advantages of the present invention will becomeapparent from the following more detailed description, taken inconjunction with the accompanying drawings, which illustrate, by way ofexample, the principles of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate the invention. Throughout thedrawings like reference numbers indicate like exemplary elements,components, or steps. In such drawings:

FIG. 1 is a block diagram of a digital camera system embodying thepresent invention;

FIG. 2 is a diagram of an embodiment of the digital camera system;

FIG. 3 is a diagram of an alternate embodiment of the digital camerasystem;

FIG. 4 is a diagram of an additional embodiment of the digital camerasystem covering a remote sensor and frame grabber camera module;

FIG. 5 is a diagram of another embodiment of the digital camera systemcovering a remote three-sensor and frame grabber camera module;

FIG. 6 is a diagram of yet another embodiment of the digital camerasystem;

FIG. 7 is a diagram of an alternate embodiment of the digital camerasystem;

FIG. 8 is a diagram of another embodiment of the digital camera system amobile docking camera and processing system;

FIG. 9 is a diagram of another embodiment of the digital camera systemcovering stereo 3D network recording and visualization;

FIG. 10 is a diagram of another embodiment of the digital camera systemcovering a mobile stereo camera and recording system;

FIG. 11 is a diagram of another embodiment of the digital camera systemcovering a multi-camera and stereo-3D system with network and broadcastinfrastructure; and

FIG. 12 is a flow chart for SiliconDVR software associated with thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

As shown in FIGS. 1-12 for purposes of illustration, the presentinvention resides in a digital camera system that captures scalableresolution, bit-depth and frame rate raw or color processed images fromone or more modular imaging modules at precise film or video rates. Thepresent invention utilizes industry standard cabling infrastructure fortransmitting either the raw sensor data or processed raw on the same ordifferent links. The present invention provides a mechanism for timingsynchronization of exposure and readout cycles from multiple imagingmodules. The present invention also provides a unified software oroperator interface to control the capture, processing andnon-destructive visualization from one or more imaging modules. Thepresent invention can optionally combine live imagery with previouslystored imagery or computer generated virtual sets, while simultaneouslyrecording the raw, broadcast format, or visualization processed imageryin its original or near original representation. The present inventiondiscloses a processor that can be used to compress the imagery with anencoder, which can generate multiple streams one for the purpose ofrecording at highest quality and optionally additional streams at lowerdata rates for remote transmission. The present invention enables therecording of one or multiple image streams using a common removablestorage device or across multiple devices for increased throughput. Therecording can make use of a single file containing the streams frommultiple imaging modules with metadata to enable the selective playbackof one or more streams. The output processing can include mixing theimagery from the multiple streams for display on standard computer orbroadcast monitoring devices or processed for output on specializedstereographic displays that require formatting and synchronization fromdual image streams. Utilizing metadata encoded in the recorded stream orgenerated thru user input during playback the relative position, colortransformation and format of the dual streams, representing the left andright eye content, can be adjusted to change the stereographic effectand depth perception on these displays.

The present invention enables reduced complexity for capturing imageryfrom one or more image modules, enables remote image sensing and framegrabbing with transmission using existing industry standard broadcastand networking infrastructure, improves storage and processingefficiency, provides increased flexibility and tools visualization,networking, analysis and mixing of prerecorded or computer generateddata, and delivers unique display modes 2D and 3D representation of themultiple streams during live, recording, playback or post processing.The present invention discloses a digital camera system that includesoptics, one or more imaging modules, a frame grabber, a processor,software, user input mechanism, a display, synchronization mechanism,networking means and storage means. In addition, a configuration isdisclosed for a portable digital camera and recording system capable ofHD, 2K and 4K stereo-3D or wide-dynamic multi-image acquisition usingtwo image sensing modules and separated image processing, recording anddisplay subsystem.

An embodiment of the present invention in the form of a digital camerasystem 20 is illustrated in FIG. 1 and described below in order toprovide an overview of the camera system 20 as well as variouscomponents of the system 20 and their respective functions. The camerasystem 20 includes an optical assembly 22 to gather light 24 from adesired scene 26. The system 20 also includes a modular imagingsubsystem 28 aligned with the optical assembly 22 to receive light 24gathered and/or modified by the optical assembly 22. The modular imagingsubsystem 28 comprises one or more imagers 30 and at least one framegrabber 32. The imager 30 captures high definition raw images at film orvideo rates for HD, 2K and 4K, cinema quality production. The imager 30can come in various forms including, without limitation, at least onepixilated image sensor unit having one or more arrays of pixels, apickup tube, a semiconductor detector or the like. The pixilated imagesensor unit can come in various forms including, without limitation, acomplimentary metal-oxide semiconductor (CMOS) active-pixel imagesensor, a metal-oxide semiconductor (MOS) active-pixel image sensor, acharge-coupled device (CCD), a contact image sensor (CIS) or otherpixilated detection devices. A single image sensor 30 may include colorfilters that are used to capture a representation of the full colorimages.

The optical assembly 22 includes optics (e.g., lenses). The system 20includes a lens mount (not shown) that interconnects the opticalassembly 22 and the modular imaging subsystem 28. The lens mount cancome in various forms including, without limitation, a fixed opticalinterface mount, an interchangeable lens optical interface mountingsystem or the like. Thus, the lens mount can provide for film or videolenses to be removably connected to the modular imaging subsystem 28.The interchangeable lens mount is a precise mounting surface and lockingmechanism, which enables field exchange of the lens mount to support theuse of a variety of industry standard lenses such as, PL, Nikon-F,Panavision, Leica, C and Canon. An interchange lens mount with anintegrated optic enables the use of B4 optics (originally designed foruse with three-sensor prism cameras) on a single-sensor based cameraunit. In the alternative, the image sensor unit may have an integratedlens.

The image sensor unit 30 includes a plurality of adjustment mechanisms(not shown) to adjust the position of the image sensor unit relative tothe optical center of lens projection and/or to adjust the co-planarityof a sensing plate (i.e., the surface which holds the sensor circuitboard of the image sensor) relative to the optical interface mount. Theimage sensor unit 30 also includes a mechanism for back focusadjustment. Any of the adjustment mechanisms can include an electronicpositioning device for remote operation for moving the sensor(s),optics, camera(s) or rig(s). In the alternative, the image sensor unit30 may be integrated with an optical beam splitter or rotating shuttermechanism to enable the use of an optical viewfinder while continuing toacquire imagery. In another alternative, an electronic display unit canbe mounted into the optical beam splitter mechanism to enable selectableoperation as an optical or electronic viewfinder or as a combinationoptical viewfinder with virtual electronic image display.

An optical device (not shown), such as an RGB prism, may be positionedin front of the optical assembly 22 so that a plurality of pixilatedsensor units 30 in the imaging subsystem 28 capture color-separatedchannels. Alternatively, a plurality of pixilated image sensor units 30and beam splitting optics may also be used to capture a wide dynamicrange representation of light 24 from the scene 26, where each pixilatedimage sensor unit 30 captures a range of scene intensity (i.e., eachsensor will have a bounded range of intensity that the sensor canaccurately detect or measure based on the capacity of sensitivity andsetting of the camera and its sensor (e.g., one cannot typically seedetails of both the Moon and the Sun in the same scene). Signals fromeach of the plurality of image sensor units 30 can be processed andcombined into a single image representing a wider range of sceneintensity than can be accomplished with a single image sensor unit 30.Each image sensor unit 30 is capable of outputting multiple readoutsfrom the at least one array of pixels with varying integration times foreach pixel of the array during a single frame time, to be later combinedto achieve a wide dynamic representation of light 24 from the scene 26.The multiple exposure readout per frame can be applied to single ormultiple image sensor unit configurations.

The image sensor unit 30 contains a time base and controller withprecision to enable audio synchronization. The pixel data is readout ofthe image sensor unit 30 at variable clock speeds or resolutions and iscaptured via the frame grabber 32. The images output from the imagesensor unit 30 may be captured by the frame grabber 32 eithercontinuously or on an intermittent basis. The image sensor unit 30 canoperate stand alone to generate and output image data over a high speeddata link to the remote frame grabber 32. The image sensor unit 30includes local inputs (e.g., sensor input, motor encoders, timecode,triggers, etc.) and can generate external sync and control signals(i.e., signal output which can be used to synchronize other cameras tothe first camera's timebase; control can be for lens, motors, lights,etc.). The image sensor unit 30 can also accept synchronization andclock source reference from the frame grabber 32 (i.e., the framegrabber 32 gets the external sync information and sends the externalsync information to the sensor or operates the sensor timing).

The image sensor unit 30 can operate in either a continuous mode or askip frame output mode. The image sensor unit 30 can also operate in areadout mode at a rate greater than the desired capture frame rate(i.e., the rate at which data is captured by the frame grabber 32),generally twice the desired capture frame rate, and can produce anassociated synchronization signal (e.g., once for every two images).This synchronization (or sync) signal can be used to instruct the framegrabber 32 to capture the intermittent or alternating frames, withspecific timing relative to the top of frame readout. The sync signalcan be output for external synchronization of additional cameras (e.g.,sensor units, camera modules, etc.). The sync signal can also bereceived by the image sensor unit 30 from external sources (e.g.,another image sensor unit 30, modular imaging systems (i.e., imager andframe grabber), a master sync generator which generates sync signals forall sensor units or camera modules). In this manner, the image sensorunit 30 can be programmed as a master to output the sync signals or asslave to receive a synch signal from external sources. An input (i.e.,command via hardware (from external signal device (keypad), or from theframe grabber 32 thru software initiation from a recorder) to the imagesensor units 30 can be used to set the operation either as a master or aSlave camera. An input can also be used to select a skip or non-skipsynchronization mode. In a skip mode, the master camera (i.e., themaster sensor unit) will only output a synch pulse at the top of frameof each frame that should be grabbed (i.e., captured by the framegrabber 32). The slave camera (i.e., the slave sensor unit) which isprogrammed for skip mode, will receive a pulse every other frame scannedfrom the master to begin a synchronized readout of a frame pair. Thedouble speed readout skip capture mode allows reduced motion blur with afaster readout and also enables the use of a mechanical shutter, such asa rotating mirror shutter of the type used on traditional film cameras,with a rolling shutter readout sensor, such as are commonly found onCMOS image sensors, wherein the image exposed during the open period ofthe mechanical shutter can be readout during the closed period.

The frame grabber 32 includes an on-board memory (not shown) to capturefull frame images (from a single or multi-exposure readout image data)or buffer the image data for transfer to a processing subsystem 34having an associated memory (i.e., a RAM part of the frame grabber 32).The frame grabber 32 can be used for image pre-processing that includes,without limitation, noise and pixel defect correction, binning,sub-sampling, black frame offset correction, multi-exposure data mixing,data packing or data coding which may be lossy or lossless.

The frame grabber 32 is able to receive input from external componentsand/or from the processing subsystem 34 to initiate specific actionswhich may include triggers for timing synchronization or eventinitiation (e.g., timing synchronization of readout and event initiationsuch as “Start recording” or “move lens”). The external inputs can bede-bounced and re-timed relative to the image readout cycle to enableprecise record start event synchronization.

The frame grabber 32 includes a time code reader (not shown) and atiming unit (not shown) for matching time reference with other systemdevices, such as an audio recorder (not shown). The time code readerobtains a time reference from a master source and the timing unit keepsthe time like a clock. The timing unit can set its reference time froman external time code source. The time code generator is a clock whichhas an output connection to allow other devices to receive the currenttime and set its clock to match it. The timing unit (i.e., the clock)may contain a charge-storing device, such as a large capacitor, whichcan enable continued operation and time keeping when an external powersource (not shown) is disconnected from the system 20. The system 20 mayalso be powered by a battery (not shown).

The frame grabber 32 may also generate outputs for controlling variousdevices including, without limitation, motors such as pan and tilts toposition the camera (i.e., the image sensor unit 30), rotating shutters(i.e., spinning mirrors like those used on traditional film cameras),slide motion stages (i.e., that slide along a rail from the left toright side of a stage along a beam) or lens motors for zoom, focus oriris, which may be used to control a stereo 3D rig or lighting such asstrobes. The frame grabber 32 can also generate synchronization signalsfor multi-camera operation. The frame grabber 32 can also receiveexternal information from sensors (e.g., positioning encoders, laserdistance sensors, microphones), rotating shutter position detectors,time code generators and positioning devices. The external informationreceived by the frame grabber 32 can be used for system control andtransmitted along with the imagery (i.e., the data from image sensorswhich can be raw or processed, codec or uncoded) for recording asmetadata or user interface feedback. The frame grabber 32 can alsoaccept audio signal input, which can be processed and mixed into theimage data stream for transmission. The audio source is another datasource. It is desirable to transmit the audio together to keep lip syncwith the images (e.g., sounds heard correspond to images of a mouthmaking those sounds).

The frame grabber 32 can be used to generate color processed RGB ortransformed YUV imagery at the time base of the readout from the imagesensor unit 30 or scan-converted for output at a different rates. Anexternal sync reference signal can be used to establish the rate andphase for the scan-converted output. The frame grabber 32 can output theimagery (i.e., a sequence of motion pictures) as standard definition,high definition (digital or analog signaling) or computer outputformats.

The frame grabber 32 can be housed with the image sensor unit 30 in ashared housing or enclosure (not shown). In the alternative, the framegrabber 32 can be remotely connected to the image sensor unit 30 inorder to allow a smaller form factor sensor unit (i.e., a camera headsized to fit into a hand, put on a goal post for sports, etc.). Combinedin the same housing, the image sensor unit 30 and the frame grabber 32(i.e., modular imaging system 28) comprise a modular camera unit 28which can operate either standalone with an output for display or can beconnected to a processing sub-system 34, capable of operating one ormore modular camera units 28. The modular camera unit 28 can beremovably docked (i.e., electro-mechanically connected) to theprocessing sub-system 34, or can be detached from the processingsub-system 34 for remote camera unit operation (i.e., remote operationof the modular imaging system 28) via wired or wireless connections. Adocking mechanism (not shown) provides electro-mechanical connection ofthe modular camera unit 28 and the processing sub-system 34 that enablesfield insertion or removal of the modular camera unit 28 from theprocessing sub-system 34. In the docked position, the modular cameraunit 28 can receive power and controls from a processor 36 comprising aportion of the processing sub-system 34 and can also transmit images(i.e., image data) and metadata to the processor 36 using the dockingconnections.

The modular camera unit 28 can communicate imagery (i.e., data fromimage sensors which can be raw or processed, codec or uncoded) using adata link and may generate an output display of the motion pictures. Thedata link or data channel between the frame grabber 32 and the processor36 can transfer the raw or processed image data at variable speeds,which may be below, at or above the readout rate of the image sensorunit 30. In the case of the capacity of the data channel being less thanthe rate of readout from the image sensor unit 30, multiple images canbe buffered in the local memory of the frame grabber 32 and imagestransmitted using a first in, first out (FIFO) basis or in a skippedframe fashion. All imagery may be buffered until the memory of the framegrabber 32 is filled and then emptied at the available data channelrate. A subset of the imagery being captured into the local memory ofthe frame grabber 32 can be transmitted to obtain a live previewdisplay. The modular camera unit 28 can generate multiple data streamswith varying bit rates that are simultaneously transmitted over a singleor multiple data links. The data transmission link from the image sensorunit 30 to the frame grabber 32 or from frame grabber 32 to theprocessor 36 may use existing infrastructure broadcast transmissionmedia, such as 1.5 Gigabit/sec or 3.0 Gigabit/sec capable linksincluding Coax and triax, wireless links, fiber optics or networkcables, such as CAT-5e, to transmit either the raw 2K or 4K image dataor SMPTE format RGB and YUV serial digital data. The raw or colorprocessed data, either coded or uncoded, can be transmitted on the samedata transmission link. The data transmission link can also be used tosimultaneously transmit the raw and processed data in either coded oruncoded formats. This enables a very high quality data set to be usedfor recording, while another data set is used for remote transmission ordisplay. The data transmission links using coax or triax can incorporatea reverse control channel modulated onto the cable, from a remoteprocessing sub-system 34. Additionally, power can be received on thetriax connection. In this way, a single cable can be used for image andmetadata transmission as well as control and power.

A modular camera unit 28 that uses a 2K image sensor unit 30 can use aat least one SMPTE 1.5 Gigabit/sec HD link to transmit 2K raw image dataat fifty (50) images per second with 10-bit per pixel data. A modularcamera unit 28 that uses a 4K image sensor unit 30 can use two or more3-Gigabit per second links to transmit 4K raw images at minimum of 23.97images per second with at least 10-bit precision. The 4K image sensorunit 30 can transmit image data at a minimum of 5.1 Gigabit/second andthe modular camera unit 28 using the 4K image sensor unit 30 cantransmit data at a minimum of 2.55 Gigabit/second. A sensor unit 30 orcamera module 28 may use a four pair network cable with up to threepairs carrying data to the frame grabber 32 and the fourth pair forreverse control and optional feedback (which may include a digitallyencoded display stream for output). An alternate configuration can usethe network cable as a single Gigabit Ethernet connection from the framegrabber 32 that can be used to transmit 12-bit raw uncompressed data atover 100 MB/sec to enable capturing of 2048 by 1152 resolution imagery(i.e., RAW data which may or may not be coded by the frame grabber 32)at up to twenty five (25) images per second, even if the sensor unit has4K resolution. In a sensor windowing mode, the frame grabber 32 cantransmit 1280 by 720 resolutions at rates faster than standard 720Pvideo and film rates, up to eighty five (85) images per second. At aresolution of 960 by 540, a rate of one hundred fifty (150) images persecond can be achieved. Similarly, other resolutions and over-crankingframes rates can be achieved within the channel limits of the link rate.Using coding methods on the data, higher resolutions and frame rates canbe transmitted, including but not limited to cinema 4K. The GigabitEthernet link may use power-over-ethernet or additional dedicated wirepairs alongside the Ethernet, to achieve a single connection for power,data and control.

Data received by the processing sub-system 34 includes, withoutlimitation, image data, metadata, control signals or the like. Themetadata may include information pertaining to the frame grabber 32, theimage sensor unit 30, camera or lens settings, scene data, or externalinputs (e.g., audio, global positioning system (GPS), Timecode or motorposition feedback. The control signals may include functions forsynchronization and event initiation such as starting and stopping ofrecordings.

The processor 36 of the processing sub-system executes reprogrammablesoftware 38 that performs image processing for visualization, analysis,or storage. The processor 36 may be either dedicated hardware or generalpurpose central processing unit (CPU), graphics processing unit (GPU) orDSP or a combination thereof.

The reprogrammable software 38 can use a touchscreen oriented userinterface, run on an industry-standard notebook computer or workstationx86 PC platforms, and use built-in communication and display ports oradditional modules can be added into the notebook computer orworkstation for additional frame grabbing, processing, storage ordisplay output functions.

The reprogrammable software 38 can perform various image processingfunctions including, without limitation, image correction,interpolation, white balance, color correction, color transformationincluding the use of three dimensional look-up tables, motion detection,object detection and classification, tracking, triangulation,calibration, color keying, image mixing, stereographic processing,anaglyph generation, indicator overlay, focus detection, exposuremetering, zooming and scaling, flipping, data packing, pattern matchingand recognition, face recognition, data rate analysis, enhancement,stabilization and compression. The image processing functions may besoftware selectable and may be combined for multiple image processingfunctions.

The image data can be stored, displayed, or transmitted. The processor36 generates a file management system for subsequent data storage. Thesystem 20 also includes a display 40 connected to the processor 36. Thedisplay 40 can come in the form of various devices including, withoutlimitation, electronic viewfinders, cathode ray tube (CRT) monitors,liquid crystal displays (LCD), organic light emitting diodes (OLED),LCOS displays or projectors, or stereographic displays, such as virtualreality (VR) goggles. The display 40 may be on-board the camera orremotely connected. The processor 36 generates a low latencyrepresentation of the scene 26 to the display 40 for user feedback andinteractive control or positioning. In the event the processingsub-system 34 is not capable of displaying full resolution, full framerepresentations, the processor 36 can send a reduced resolution orreduced frame rate to the display 40 in order to maintain the lowlatency.

The processor 36 generates one or more outputs for display with imagedata, status and setting information or on-screen menus that are usablefor a touch-screen user interface. The output can also be set to displayimage data with or without image processing in a full screen displaymode without the status or operator control information, for projectionor broadcasting. In this mode, it may still be possible to mixadditional imagery, graphics and overlays for use in the transmission orrecording. These features may include sports image annotation,advertisement insertion, animations, keying, virtual set integration ormulti-channel streaming content mixing. In multiple display outputconfiguration, one monitor or display 40 may be used for user interfaceand additional outputs used for full screen display with different imageprocessing functions applied to the user images.

The processor 36 can output raw image data, image processed data ormetadata in any combination for storage, transmission or display. Theprocessor 36 can also generate outputs and external controls such aslighting controls, cooling system control, power management, motorpositioning, lens controls, time code, device synchronization,multi-camera synchronization, calibration stimulus, tactile feedback,status indicators and audio.

The processor 36 monitors the temperature and recording status of theprocessing sub-system 34 and can automatically adjust a cooling system(not shown) that cools the processing sub-system 34. The cooling systemcan include a fan that dissipates heat. The processor 36 adjusts thecooling system in various ways including, without limitation, reducingfan speed, to lower the ambient noise levels generated by the camerasystem 20.

The processor 36 can accept input from a user (via a user interface) orthe frame grabber 32 to perform specific tasks. Various input mechanisms42 may include, without limitation, a computer mouse, a pointing device,a touch screen input, direct digital signaling or network commands. Thespecific tasks may include, without limitation, start or stop recording,initiate playback, adjust the settings of the frame grabber 32 and/orthe image sensor 30, or select image processing or display modes.

The system 20 also includes a storage device 44. The storage device 44can be either internal or external and either fixed or removable.Examples of various storage devices that may be used include, withoutlimitation, non-volatile memory, flash memory, a magnetic hard drive, anoptical disk and tape. For the purposes of external storage, display orprocessing, the processed or raw data may be externally transmitted. Thetransmission methods may include, without limitation, USB,Firewire/IEEE1394, SATA, Ethernet, PCI, PCI Express, Camera link, HDMI,DVI, HD-SDI, Displaylink, Infiniband, Wireless or Fiber optic link.

A particular configuration of a digital camera system 20 that usesmultiple image sensor units 30 (via one or more modular camera units 28)input into a single processing sub-system 34 may be used for capturingmultiple image simultaneously with the ability to synchronize thesources (i.e., sensor imaging units or camera modules), coordinatecontrol and combine image processing, recording, display, storage andcommunication. This multiple camera configuration can be used forprocessing 3D stereographic and immersive scenes. The imagery (i.e., theRAW image data) and metadata (i.e., audio, positioning, timecode, etc.)from this multiple camera configuration can be recorded on a singleremovable storage medium or to independent storage devices in asynchronized fashion to enable simpler post-processing and display. Thecombined imagery can be outputted to specialized displays such asstereographic LCD monitors, 3D or spherical projection systems and VRgoggles.

The system 20 includes software 38 stored on a memory and running on theprocessor 36. The software 38 provides the user with the ability toobtain stereo imaging. An imaging software program 38 provides controlof single and stereo image sources (i.e., imagery) with synchronizedimage capture, frame grabbing, processing, metadata capture, display,coding, recording and playback, using a single user interface. The imagesources (i.e., imagery) can be from image sensor units 30 or cameramodules 28 capable of capturing high definition raw images at film orvideo rates for HD, 2K and 4K cinema quality production. As discussedabove, the image sensor unit 30 may be based on at least one CMOS, CCDor other pixilated detection device that contains a time base andcontroller with precision to enable audio synchronization. The system 20can record the sound or the timecode for the audio can be synchronizedwith the timecode for the images. A user can record the audio with theimagery or the timecode which is associated with the audio in anotheraudio recording device which also records timecode and then tied backtogether during editing or post production process.

The software 38 running on the processor 36 may automatically detect thepresence of an image sensing unit 30 or a camera module connected to aNetwork or to a hardware frame grabber 32 to determine the camera module28 or sensor unit identification and its image capture or processingcapability. Upon identification, the software 38 can load imagecalibration data from a storage device 44 or can initiate a calibrationprocess, which can extract data from the connected camera module(s) 28including, without limitation, pixel-by-pixel black level, gains,shading, and defect pixels.

The software 38 running on the processor 36 adjusts settings of thecamera module 28 and/or image sensor 30. The software 38 adjusts varioussettings including, without limitation, resolution, frame rate,exposure, gains and stereo sync source master or slave. The software 38can program a camera module 28 or an image sensor unit 30 as a masterthat uses an internal sync and outputs the sync signals or as slave thatreceives a sync signal from external sources (e.g., another cameramodule 28 or image sensor unit 30 acting as a master). The software 38can be used to set operation of the image sensor unit 30 in a continuousor skip frame output mode and to instruct the frame grabber 32 tocapture the intermittent or alternating frames, with specific timingrelative to the top of frame readout.

The software 38 running on the processor 36 can be used for controllingcamera and optical positioning devices on a 3D stereo rig for stereoeffect adjustment such as focus, iris, inter-ocular distance andconvergence. The software 38 running on the processor 36 may alsocontrol a positioning system on which the 3D rig is mounted. Thesoftware 38 running on the processor 36 can capture metadata such as rigmotor position data, timecode, lens and optics settings and camerasettings.

The software 38 running on the processor 36 can perform image and stereoprocessing functions in any combination of general purpose processor ordedicated hardware, RISC arrays or DSP's. The image and stereo functionsmay include, without limitation, image correction, interpolation, whitebalance, color correction, color transformation including the use ofthree dimensional look-up tables, motion detection, object detection andclassification, tracking, triangulation, calibration, color keying,image mixing, stereographic processing, indicator overlay, focusdetection, exposure metering, zooming, scaling, warping, flipping, datapacking, pattern matching and recognition, face recognition, data rateanalysis, enhancement, stabilization and compression. The imageprocessing functions may be software selectable and may be combined formultiple image processing functions.

The software 38 running on the processor 36 can perform compression andcan employ a full-frame temporal Wavelet transform codec to eliminate“block artifacts” that are often present when using DCT compression. Thesoftware 38 may have scalable precision to operate on data from 10-bitand higher, with optimized arithmetic precision based on sourceresolution, and scalable resolution to support a variety of formatsincluding HD, 2K and 4K. The software 38 can use constant-quality,variable bitrate (VBR) compression that allows compression rates to risedynamically for more complex scenes, and allows compression rates todynamically fall for less-complex scenes. The codec can support Rawpixel data, RGB or YUV data formats. The codec can combine data fromimage, audio and metadata and streaming metadata in headers of files andwithin groups of pictures (GOP) for use in decoding and editing. Thecoded data can be encapsulated into industry standard format filecontainers such Audio-video Interleaves (AVI), Quicktime (MOV). Inplayback, the codec, which may be on the same software platform or partof a post-production software program, can adaptively select to decodehierarchical resolution data, inherent in the wavelet transform, toenable real-time, multi-stream editing performance in software onstandard PCs, without the need for specialized hardware. The compressionor coding method may be software selectable for each recording andstreaming function. The software 38 can also capture and record imageryas uncompressed data at various bit depths.

The software 38 running on the processor 36 can code stereo streams asindependent full-frame streams or can pre-combine the imagery into asingle larger image or interleaved sequence for coding as a singlestream. The metadata contained in the stream can be used to indicate theleft and right image source and allow playback and editing of the stereofile as a single video source, yet displaying either source individuallyor as a mixed representation for stereographic display.

The software 38, either in a mobile stereo recorder of the typedescribed below or on a separate playback system, can retrieve andrender the recorded imagery into a sequence of raw images along withmetadata as industry standard Digital Negative (DNG) files or as fullyprocessed RGB or YUV images based on metadata stored within the imageand data stream or in an associated container file, in formats such asDPX and TIF. The software 38 may allow modification to the associatedmetadata streams effects on the retrieved images. The method for debayeralgorithm can be selectable or accomplished thru replacement of softwaremodules.

The software 38 running on the processor 36 can generate processeddisplay imagery from live or playback sources on single or dual outputs.The colorized raw or processed image data and metadata can be sent to ahost (i.e., a separate computer which is not the mobile stereo recorder)for additional display, processing, recording or transmission. Thesoftware 38 can flip and mirror display imagery to enable a viewingsystem with two displays on a 3D beam splitter viewer. The display imagedata can be formatted and scaled for standard definition or highdefinition displays 44. For bayer image sources, the software 38 can beused to select the demosaic method, based on the available processingcapability. The processed imagery may include generating stereographicdisplays including dual-streams image mixing, anaglyph, over-under,side-by-side, sequential switching and other modes, which may assist inperceiving the potential 3D stereo.

The overlap or mix from the dual image streaming sources may be adjustedrelative to each other to change the stereo effect. The repositioningand adjustment may include translation, rotation and warping. Theresulting adjustment done thru the user interface becomes anothermetadata source, while allowing the full-size non-adjusted original datato be transmitted or recorded.

The software 38 running on the processor 36 can operate in aclient-server configuration with remote control over a wired or wirelessnetwork. The software 38 can accept a trigger to initiate synchronizedstart and stop recording. Client software can request and receive datafrom a server on the network where the data comes in various formsincluding, without limitation, single images, stereo image, streamingimages, audio, time code, camera settings, server settings, projectsettings, color look up tables and other metadata. The client softwarecan send the same or modified versions of the data back into a camerasystem which also has the processing, such as the mobile stereorecorder, and effect changes on the live or recorded data.

The software 38 running on the processor 36 can execute a timed motionand recording event sequence, which may include, without limitation,start recording, continuous adjustment of stereo rig positioning andlens settings such as programmed slew rates, target positions andpauses, record speed changes and event and timer triggers.

The software 38 running on the processor 36 may have a mode forcalibration of a display device 44. In the calibration mode, thesoftware 38 can generate test pattern outputs for stimulus and theresponse values can be measured using a connected optical sensor. Asequence of stimulus and response measurement values can then be used tocreate a modification to the imagery sent to the display 44, such asusing 3D Look-up-tables applied to the raw data or used to modify thesettings on the hardware used to generate the output.

The software 38 running on the processor 36 can combine the stereoimagery in a virtual studio which takes a video image of live scene shotagainst a keying color background or stage and composite them against acomputer-generated 3D environment to create the illusion that the liveactors are actually inside and interacting within a virtual world. Thesoftware 38 can switch between multiple stereo sources, have selectableimage and audio stream mixing or delaying, chromakeying, and renderer 3DGraphic real-time. The software 38 can provide trackless camera controlwhere the camera's motion and switching between shots are accomplishedby manipulating the 3D virtual set itself rather than by manipulatingthe real, physical cameras. Within the Virtual Studio, movements likecomplicated pans, swoops, and tilts are then possible because the camerais not actually physically moving—the 3D set is. The software 38 canhave a pre-defined track assigned by 3D design or manipulated through anexternal input device such as a joystick. The software 38 can makepositional adjustments of the stereo rig (i.e., a physical package withtwo sensor units or camera modules) synchronized with changes in thevirtual set further enhancing the stereo perception. The combined outputof the stereo sources and virtual set can be displayed for liveproduction and can be encoded for streaming and recording.

The software 38 running on the processor 36 can operate on independenthardware platforms for increase processing power, where capture,processing, switching and effects, such as virtual sets, streaming andrecording can be distributed and controlled via network, yet canmaintain synchronized recording events.

The software 38 running on the processor 36 records on a singleremovable storage medium or to independent storage devices in asynchronized fashion with common file naming conventions, to enablesimpler stereo post processing, editing and playback.

Various components of the processing sub-system 34 of the system 20 canbe embodied in a single housing that acts as a mobile stereo recorder46. The mobile stereo recorder 46 executes the software 38 of the camerasystem 20 and can capture, process and record synchronized imagery fromat least two image sensor units 30 or camera module source playbackusing a single user interface.

The recorder 46 includes a battery voltage input power supply withgigabit Ethernet for image and data communication. The streaming datacan then be processed by a host computer, such as a single or multi-corex86 cpu with a graphics processing unit (GPU) for display and haveinterfaces for removable storage which may include IDE, USB, Network andSATA.

The mobile stereo recorder 46 may also use an additional multi-inputframe grabber processing system. The frame grabber 32 may use FPGAdevices and scalable massively parallel RISC Processor Arrays. The RISCarray processor may use architecture of brics, which contain multiplecompute units, such as Streaming RISC units and streaming RISC unitswith DSP extensions, and memory RAM units. The RAM Units can streamaddresses and data over channels. These channels can be word-wide andrun at 10 Gigabits per second or higher. These processors can execute anoperation, do a loop iteration, input from channels, and output to achannel every cycle. These brics can connect by abutment throughchannels that cross bric-to-bric. The compute units and ram unit can bearranged so that, in the array of brics, there are contiguous computeunits and contiguous ram units. The array processor can have aconfigurable interconnect in hierarchical fashion with several levels ofhierarchy.

The frame grabber processing system may be capable of capturing stereoimage data from multiple high-speed serial digital transmission links,such as 1.5 Gigabit and 3.0 Gigabit HD-SDI, HSDL, Easylink andCameralink. The image data may be from image sensor units 30 or cameramodules 28 in raw pixel data, color processed RGB, color-processed YUVin coded or uncoded formats. The image data may also come from broadcastvideo or computer sources.

The frame grabber 32 can be capable of capturing 4K image data at aminimum of 4096×2180 from an image sensor unit 30 at a minimum 5Gigabit/sec or from a 4K camera module 28 at a minimum of 2.55 Gbit/secof raw data.

The frame grabber processing system may be capable of stereo displayoutputs. Each output may be capable of displaying live or processedimagery from the sensor units or camera modules. The stereographicoutputs may include stereo visualization video processing and signalingto drive dual displays or displays requiring mixed streams, includedsynchronization data for shutter glasses controls.

The mobile stereo recorder system 46 can execute programmable code 28for image and stereo processing functions. The processing functions canbe done by the frame grabber processor alone or in combination with thehost computer and graphics processor unit. Playback can be done on thehost or in combination with the frame grabber processing system, wherethe imagery can also be output for display. The processing and controlfunctions of the mobile stereo recorder 46 may be remotely controlledfrom another system via wired or wireless network or other input device,such as touchscreen keypad with serial communication.

The mobile recorder 46 can use a single removable storage magazine 44,which may contain at least one storage media unit. The removable storagemagazine may use at least one SATA interface. The storage device 44 mayinclude a Raid controller, with drive carrier selecting the RAID storagemethods, such as RAID-0 or RAID-1.

The mobile recorder 46 can be contained in an ergonomic package whichenables it to mount on a stereo camera stabilizing platform, such as asteadicam and MK-V Revolution System, along with stereo image sensorunits 30 or camera modules 28, which mechanically isolates the movementof the camera rig (i.e., the platform which has the sensor unit(s) orcamera modules or camera with recording system) from that of the cameraoperator, providing a very smooth shot even when the operator is movingquickly over an uneven surface.

In another embodiment of the present invention, as seen in FIG. 2, adigital camera system 50 includes an optical assembly 22 to gather light24 from a desired scene 26. The system 50 also includes a modularimaging subsystem 28 aligned with the optical assembly 22 to receivelight 24 gathered and/or modified by the optical assembly 22. Themodular imaging subsystem 28 comprises at least one imager sensing unitor imager 30 and at least one frame grabber 32. The modular imagingsubsystem and the frame grabber are in a shared housing 52 and comprise,in part, a camera module 54.

The optical assembly 22 includes optics 56 (e.g., a Carl Zeiss Super 16Ultra Prime lens) connected to the camera module 54 at an appropriateback focal distance using an interchangeable lens optical interfacemounting assembly 58 that includes an optical low pass filter (OLPF) 60(e.g., a P+S Technik PL lens Interchange mount with Sunex Optical Lowpass filter). Alternatively, the mount may be a P+S IMS InterchangeMount with calibrated back focus and sensor co-planarity adjustmentmechanism. The interchangeable lens optical interface mounting assembly58 is a precise mounting surface and locking mechanism, which enablesfield exchange of the lens mount to support the use of a variety ofindustry standard lenses such as, PL, Nikon-F, Panavision, Leica, C andCanon.

The modular imaging sub-system 28 comprises an HD/2K or 4K CMOS imagesensor unit 30 and a frame grabber and controller 32. One example of animage sensor unit 30 is an Altasens 4562 2K and HD CMOS system-on-chip,Xilinx FPGA, Microchip PIC micro controller, Linear LT series Linearregulators, IDT ICS-307 programmable clock and Fox 924 temperaturecontrolled crystal oscillator. Another example of an image sensor unit30 is an Altasens 8472 Quad HD and 4K format capable sensor. An exampleof a frame grabber and controller 32 is a Pleora iPort with FPGA, RAMBuffer and Gigabit Ethernet connectivity with serial ports and GPIO forprogrammable device control and communication. An example of a cameramodule 54 comprising an integrated imaging sub-system 28 in a housing 52is an SI-2K MINI digital camera from Silicon Imaging, Inc. The SI-2KMINI includes an Altasens 4562 CMOS imaging sensor unit and a Pleoraframe grabber. The SI-2K MINI is capable of capturing 2K (2048×1152),1080P HD (1920×1080), 720P (1280×720) AND 540p (960×540) resolutionmotion pictures. The SI-2K MINI can operate at various film and videorates including 23.97, 24, 25, 29.97, 30, 50, 59.9 and 60 frames persecond. The SI-2K MINI has local RAM buffer 62 to capture images athigher rate than the channel capacity and can buffer frames and transmiton a skip frame basis. Minimum Resolution for 2K is 2048×1080 andminimum film rates would be 23.976, except for special time lapse shoot.For 4K shooting, a minimum 4096×1714. A film rate is approximately23.976 frames per second (i.e., 24 frames per second) and video rate is25-30 frames per second.

Channel bandwidth between image sensing unit and frame grabber andbetween frame grabber and processor is sufficient for transmission toenable full resolution motion picture raw image or data processing. Thismeans the pipe to move the data from the sensor to the frame grabbermust be fast or wide enough to carry all the raw pixel data as it isreading out of the sensor at the film and video rates. For example, tomove 2K image, which has 3.3 MB per frame at 48 frames per second wouldrequire almost 200 MB/sec throughput from the sensor to the framegrabber. Often, readout from the sensor is at 2× the speed than neededto capture in the frame grabber 32. The frame grabber 32 then only needsto move 24FPS or 100 MB/sec to the PC for processing. It is reasonableto have the frame grabber 32 do lossless coding of the data to achieve a2:1 data reduction, which would get the frame grabber to host linkbandwidth at 50 MB (hence a minimum of 48 MHz as Intel 4.

The system 50 includes a record start/stop button 64 (e.g., a momentarymechanical switch) electro-mechanically connected to the camera module54 along with power and sync input and output wiring 66 using aconnector (e.g., an 8-Pin Lemo FGG.1B.308.CLAD52 connector). An outputsignal is a light emitting diode (LED) (not shown) electro-mechanicallyconnected to the iPort GPIO that illuminates when recording is activeand un-illuminated when recording is non-active.

The system 50 further includes a plurality of motors and sensors 68(e.g., a C-motion lens control system, a Preston Motor and controlsystem, etc.) that act as adjustment mechanisms to adjust the positionof the image sensor unit relative to the optical center of the lensprojected image circle and/or to adjust the co-planarity of the sensingplate upon which the image sensor pixel circuit board rests relative tothe optical interface mounting assembly 58. The sensor is mounted behindthe lens and can be adjusted for flatness, centering and rotation. Anyof the adjustment mechanisms can include an electronic positioningdevice for remote operation.

The system 50 further includes a laptop notebook computer 70 connectedto the camera module 54 by a cable (not shown) (e.g., a CAT-5e Ethernetcable) through a Network 72. The cable is connected to the camera module54 by a connector (e.g., a 12-pin LEMO FGG.2B.312.CLAD52 Connector). Oneexample of the notebook computer 70 is a Dell M90 with a Marvell YukonGigabit Ethernet Expresscard for camera connectivity. On-board wired andwireless Ethernet ports of the notebook computer provide remoteconnections streaming, internet connectivity and control.

A further embodiment of the present invention is illustrated FIG. 3,where a digital camera system 80, similar to the digital camera system50 described above, includes a camera module 54, as described above inrelation to FIG. 2, that has an additional capability for the framegrabber 32 to process raw video into a live video output. The additionalframe grabber processing section comprises a data link, sync and controlunit 82 (e.g., an Altera FPGA with Lux Media Plan HD-1100 raw colorprocessing core, dedicated clock reference for 74.25 and 74.1758 MHz,Gennum GS Series Serializers and cable drivers for 1.5 or 3 Gbit/secHD-SDI, IDT Dual Port FIFO for external time base synchronization orretiming, SSRAM, Flash and Analog Devices RAMDACs for SVGA Output)connected to the sensor imaging unit 30 and the frame grabber 32. Theprocessing core of the frame grabber 32 can convert raw pixel data intoRGB data using demosaic, image correction and color conversion. Theprocessing core can also convert between RGB and YUV spaces and outputeither 4:4:4 or 4:2:2 data. The same data link which carries the colorprocessed data from the frame grabber 32 to the viewing and recordingsystem can also be used to transmit the raw data.

The system 80 further includes an input unit 84 electro-mechanicallyconnected to the camera module 54 and a plurality of motors and sensors68 that are also electro-mechanically connected to the camera module 54.The input unit 84 includes audio sources, motion control and a time codegenerator. An Audio codec (not shown) with preamps can receive line ormicrophone level audio input to mix into the data stream. The time codegenerator can be an Ambient ALL-601. The plurality of motors and sensors68 are used, in part, to control the lens (e.g., a C-motion lens controlsystem) act as adjustment mechanisms to adjust the position of the imagesensor unit 30 relative to the optical center of the lens mount and/orto adjust the co-planarity of the sensing plate (i.e., the surface orcircuit board which holds the image sensor in the correct position)relative to the optical interface mounting assembly 58.

The system 80 also includes a display 86 (e.g., a Cine-Tal CineMage LCDMonitor, a Lite-Eye LE-450 OLED viewfinder or the like) for viewingimage data output from the camera module 54. Image, metadata andcontrols touchscreen user interface can also be shown on this display.

The system 80 additionally includes a docking recorder 88. Onecommercial available example of a docking recorder is a SI-2K availablefrom Silicon Imaging, Inc., running SiliconDVR software, with remotecamera module 54 (e.g., an SI-2K MINI) with Live Video outputprocessing. The docking recorder 88 includes USB ports to connect thedocking recorder 88 with a number of photometric measuring devices(e.g., a colorimeter which measures intensity at different lightwavelengths) which can is placed on a display 90 of the docking recorder88 to create calibration profiles. The calibration profiles serve to getaccurate settings independent of monitor adjustments (e.g., the user mayhave turned a color hue knob on the monitor) such that orange will beorange and not orange-red. The camera module 54 can beelectro-mechanically docked with the docking recorder 88.

An additional embodiment of the present invention is illustrated FIG. 4,where a digital camera system 100, similar to the digital camera systems50 and 80 described above, includes a camera head module 102 comprisingan optical assembly 22, an image sensor unit 30 and a data link, syncand control unit 82 that is located remotely from a frame grabber unit104 comprising a frame grabber unit 32 and a sync and control unit 82.

The remote camera head module 102 uses an HD/2K or 4K CMOS image sensorunit 30 (e.g., an Altasens 4562 2K and HD CMOS system-on-chip, anAltasens 8472 Quad HD and 4K format capable sensor), a Xilinx CPLD forsync and sensor timing control, a Microchip PIC micro controller, LinearLT series Linear regulators, an IDT ICS-307 programmable clock and a Fox924 temperature controlled crystal oscillator. The camera head module102 also includes a National Channelink LVDS 28:4 DS90 serialization andNational LVDS receiver and driver for serial communication and triggerinput and output. The optical assembly 22 includes a lens mountcomprising a back focus adjustable c-mount (e.g., a P+S IMS InterchangeMount with calibrated back focus and sensor co-planarity adjustmentmechanism) and an optic (e.g., a Linos MeVis c-mount lens, a P+Sinterchange B4 optical mount and Zeiss Digiprime Lens or the like).

This remote camera head module 102 is commercially available as either aSI-2K MICRO-CL or SI-1920HD-CL with 4562 sensor from Silicon Imaging,Inc. An alternate link configuration uses an National EasyLinkDS32ELX0421 for transmission over CAT5e and Coax and triax. Another linkconfiguration uses FPGA with built-in serializer logic and eitherNational or Gennum cable drivers. A further link configuration uses atriax cable to transmit image and associated data, receive control datavia demodulation, and receive power. If power is applied locally to theimage sensor unit 30, a coax cable can then operate on coax.

An Altera FPGA Serializes the sensor data output (i.e., the data fromthe sensor or from the analog digital converter which samples the pixelphotosite) and uses at least one Easylink or cable driver for the datalink. One configuration uses a Cat-5 with up to 3 pairs for transmittingthe serialized data to and from the sensor unit and one pair for power.The frame grabber 32 and processing unit (i.e., the processing unitwhich can perform video processing for display) are described above withrespect to the SI-2K MINI with Video processing where the image sensorunit 30 is remotely connected. One connection output from the framegrabber unit 104 drives an HD-SDI display 106 while another connectionoutput from the frame grabber unit 104 outputs raw pixel data 108,either coded or uncoded.

The camera head module 102 is connected via a network 72 for remotecontrol and setup. The camera head module 102 can also be connected to aMacbookPro notebook computer 110 running bootcamp and WindowsXP. SiliconImaging SiliconDVR software is used for camera control, processing,display and recording. Alternatively, the camera head module 102 can beconnected to an SI-2K docking recorder 88 running SiliconDVR software.

The system 100 further includes a plurality of motors and sensors 68that are also electro-mechanically connected to the camera head module102. The plurality of motors and sensors 68 are used, in part, tocontrol the lens (e.g., a C-motion lens control system) act asadjustment mechanisms to adjust the position of the image sensor unit 30relative to the optical center and/or to adjust the co-planarity of thesensing plate relative to the optical assembly 22.

An embodiment of the present invention is illustrated FIG. 5, where adigital camera system 120, similar to the digital camera systems 50, 80and 100 described above, includes a camera head module 122, similar thecamera modules 54 and 102 described above, comprising an opticalassembly 22, at least one pixilated multi-sensor image sensor unit 124and a data link, sync and control unit 82 that is located remotely froma frame grabber unit 104 comprising a frame grabber unit 32 and a syncand control unit 82.

The remote camera head module 122 further comprises an optic 126 (e.g.,a beam splitter) to split light 24 in multiple directions. For example,one particular configuration of the camera head module 122 comprises twosensors 128 (each sensor 128 having an array of pixels) and a beamsplitter 126 for stereo 3D or wide dynamic image capture. Each sensor128 can comprise either a monochromatic pixel array or a color-filteredpixel array. Each of the sensors 128 can be controlled and movedmechanically or, alternatively, each of the sensors 128 can beelectronically windowed to move the readout region of the particularsensor 128. One of the two sensors 128 can be replaced with an opticalviewfinder port, allowing simultaneous thru-the-lens focusing anddigital capture. In an alternate configuration, the camera head module122 comprises an RGB prism 126 and three monochromatic sensors 128 (asseen in FIG. 5), one for each color channel. Additional prism ports areadded to achieve additional targeted wavelength or full spectrumillumination range capture. One connection output from the frame grabberunit 104 drives an HD-SDI display 106.

The camera head module 122 is connected for remote control and setup.The camera head module 122 can also be connected to a MacbookPronotebook computer 100 running bootcamp and WindowsXP. Silicon ImagingSiliconDVR software is used for camera control, processing, display andrecording. Alternatively, the camera head module 122 can be connected toan SI-2K docking recorder 88 running SiliconDVR software.

The system 120 further includes a plurality of motors and sensors 68that are also electro-mechanically connected to the camera head module102. The plurality of motors and sensors 68 are used, in part, tocontrol the lens (e.g., a C-motion lens control system) act asadjustment mechanisms to adjust the position of the image sensor unit124 relative to the optical center and/or to adjust the co-planarity ofthe sensing plate relative to the optical assembly 22.

FIG. 6 illustrates an additional embodiment of the present invention,where a digital camera system 130, similar to the digital camera systems50, 80, 100 and 120 described above, includes an optical assembly 22, acamera head module 132, similar the camera modules 54, 102 and 122described above, connected to the optical assembly 22. The camera headmodule 132 is remotely located from a processing, storage and displaysystem 134. The camera head module 132 can be in the form of an SI-2KMINI, as described in detail above and shown in FIG. 2, where the imagesensing unit 30 comprises a Kodak 2 k×2 k CCD and the processing,storage and display system 134 comprises a Dell Precision Workstationwith Intel Core 2 Duo processor, Intel Pro/1000 Gigabit Multiple PortNetwork Interface card, Nvidia Quadro Graphics GPU, CRU Removable DP-25SATA Cartridge system and LCD display with 1920×1200 native resolution.The camera head module 132 (i.e., SI-2K MINI) is powered by a 12 VDCsupply or Li-ion battery, such as an Anton Bauer Dionic 90 and isconnected to the processing, storage and display system 134 usingGigabit Ethernet over CAT5e or thru Cisco fiber optic links and routers.The 12-bit raw imagery and metadata is transmitted between the camerahead module 132 (i.e., the SI-2K MINI) and the processing, storage anddisplay system 134 (i.e., the Dell Precision Workstation) at minimumsustained data rates of 80 MB/sec for 2K DCI 2048×1080 capture, control,visualization, recording and communication. A laptop notebook computer136 (e.g., a Dell M90) is connected to the processing, storage anddisplay system 134 (i.e., the Dell Precision Workstation) via wired orwireless network for remote control, viewing, playback and metadatamanagement.

As seen in FIG. 7, another embodiment of the present inventionillustrates a digital camera system 140, similar to the digital camerasystems 50, 80, 100, 120 and 130 described above, showing a modularimaging sub-system 28 that comprises, in part, a camera head module 132that is docked with a processing, storage and display system 134. Thecamera head module 132 is commercially available as the SI-2K MINI, asdescribed above in other embodiments. The processing, storage anddisplay system 134 includes a raw image processing and communicationunit 36 (e.g., a computer based on an Intel Core 2 Duo T7600 2.33 GHzmulti-core processor with dual channel DDR2 667 MHz RAM, Intel PRO/1000Gigabit Ethernet and Intel GMA950 Integrated GPU with dual HD videooutputs, such as the MEN F17 Modular Compact PCI system). Theprocessing, storage and display system 134 also includes a user inputdevice 42 (e.g., an Interlink Micromodule mouse pointing module)connected via USB to the processor 36. The processing, storage anddisplay system 134 also includes another user input device 42 in theform of a Soundevice USBPre for audio input. The processing, storage anddisplay system 134 further includes a battery 138 (e.g., an Anton BauerHytron140 Lithium Ion connected via the Anton Bauer Gold Mount plate orexternal 4-pin Neutrik XLR) and a DC battery input power supply 118(e.g., an Opus Solutions DCA7.150 150W DC-DC converter) which deliversregulated 12VDC power for peripherals and 5VDC to power the processingand communication subsystem 134.

The processing and communication subsystem 134 further includes adisplay device 40 (e.g., a Xenarc 700TSV LCD Monitor with touchscreen(where the touch screen also provides an additional user input via a USBinterface) or Kopin Cyberdisplay with DVI interface are display devices.

The imaging software 38 used in the processing and communicationsubsystem 134 is the Silicon Imaging SiliconDVR program running underMicrosoft Windows XP. Other software tools used by the SiliconDVR forimage processing include Microsoft Direct-X graphics, CineFormRAW Codecand Iridas Speedgrade 3D look-up-table software.

The processing and communication subsystem 134 additionally includesremovable digital storage 44 (e.g., a CRU DP-25 SATA RAID Frame, KingwinKF-25 mobile rack with Seagate 2.5″ Hard drive, Crucial Flash device, or32 GB Intel SLC Solid State Drive SSD Device with 150 MB/sec writespeed). The system 140 includes a laptop computer 142 (e.g., a DellPrecision M90) and a Network recording, editing and visualization system144 (e.g., a Dell Precision 390 workstation) and a wireless computer 146(e.g., a motion computing LE1700). The laptop computer 142 and theNetwork recording, editing and visualization system 144 are eachconnected to the removable digital storage 44 of the processing andcommunication subsystem 134 via a wired or wireless network 148. Thewireless computer 146 operates over a 802.11 wireless network which isusing D-link wired or wireless routers or USB wireless networkingdevices to connect to the other components of the system 140.

An additional embodiment of the present invention is illustrated FIG. 8,where a digital camera system 150, similar to the digital camera systems50, 80, 100, 120, 130 and 140 described above, comprises an image sensorand frame grabber camera module 152, similar the camera modules 54, 102,122 and 132 described above, including a display processor 154, embeddedtime-code reader 156 and motion control 158, where the camera module 152is docked with the modular processing, storage and display system 134.The system 150 includes an optical assembly 22. The camera modulesincludes a CMOS image sensor unit 30 (e.g., an Altasens 4562 HD/2Ksensor or 8472 Quad HD and 4K format capable sensor). An Altera FPGAserializes the sensor data output and uses at least one Easylink orcable driver for a data link. One configuration uses three or more pairfor transmitting the 4K serialized data to and from the image sensorunit 30 and one pair for power. The camera module 152 is connected tothe docking recorder 88 by a connector (e.g., a 12-pin LEMOFGG.2B.312.CLAD52 Connector). An Additional LEMO connector is docked.

The optical assembly 22 includes optics 56 (e.g., a P+S interchange B4optical mount and Zeiss Digiprime Lens) and a lens mount (e.g., a P+SIMS Interchange Mount with calibrated back focus and sensor co-planarityadjustment mechanism).

The modular processing sub-system 134 comprises a computer based on anIntel Core 2 Duo T7600 2.33 GHz multi-core processor with dual channelDDR2 667 MHz RAM, Intel PRO/1000 Gigabit Ethernet and Intel GMA950Integrated GPU with dual HD video outputs, such as the MEN F17 ModularCompact PCI system. The processing sub-system includes at least one userinput device 42 (e.g., an Interlink Micromodule mouse pointing moduleconnected) via USB to the processor 36.

A frame grabber processing unit 32 is connected to the MEN F17 ModularCompact PCI system using a PCIe bus interface daughter card, whichcontains Altera FPGA and at least one Ambric AM2045 scalable massivelyparallel RISC Processor Array, and DDR2 400 RAM with 4-lane PCIeinterface. The serialized data from the camera module 152 is receivedthru National Easylink or HD-SDI receiver deserializers and thenconverted thru the FPGA to parallel data for input into the Ambric framegrabber, where it can then be processed and color corrected andcompressed using a Wavelet Codec, such a Cineform or JPEG2000. The samereceivers can alternately be used for multiple camera module input. Theraw data or coded data can be accessed by the Intel Core 2 duoprocessor, where the processor can perform additional image processing,visualization output or recording. An additional Ambric processor can beused for video format conversion and re-timing for live display inHD-SDI or computer format. The processed video can also be sent back ona data pair back to the camera module 152 for displays 40 connecteddirectly to the camera module 152 and where there is insufficientprocessing or space for performing the processing in the camera moduleitself. An alternate display section uses the Lux Media Plan Videoprocessing core and memory buffers to produce scaled and retimed livevideo in SD, HD and 2K outputs. The processing section can support dualoutputs for stereo display.

Additional computers connected to the system can be used to adjust andmanage metadata which can adjust settings such as visualization modes,used for recording of image data streams or used for processing ofstreams into live outputs for display on local monitors or to live orplayback broadcast feeds. The computers can also be used to capturephotometric sensor data to adjust the display calibrations which canbecome part of the visualization image processing. A laptop notebookcomputer (e.g., a Dell M90) is connected to the processing, storage anddisplay system 134 (i.e., the Dell Precision Workstation) via wired orwireless network for remote control, viewing, playback and metadatamanagement.

FIG. 9 illustrates an alternate embodiment of the present inventionwhere a digital camera system 180, similar to the digital camera systems50, 80, 100, 120, 130, 140 and 150, comprising multiple camera modules54 (e.g., SI-2K MINI) and multiple processing, recording and displaysub—systems connected via network 182. The camera modules 54 (e.g.,SI-2K MINI) are configured with an optical mount (e.g., a P+S Technik B4optical mount and Angenieux 19×7.3 BESSDE HR Motorized Lens) and arepowered from an external 12VDC supply or Anton Bauer Hytron 140 Goldmount systems.

A first pair 186 of camera modules 54 (e.g., SI-2K MINI) are mounted for3D stereo capture of a common scene thru an Edmund Scientific NT46-584optical beam splitter 184. The two camera modules 54 are cabled togetherand are synchronized to capture at the same rate and to begin scanningwith a common frame start time.

A second pair 188 of camera modules 54 (e.g., SI-2K MINI) are mountedadjacent to each other for 3D stereo capture on a synchronizedmulti-axis pan, tilt, theta and slide motorized stage system withNewport controllers 190. The motor positions are operated from software38 either manually, in response to image processing or as a pre-setsequence triggered by events or timers. The system 180 also includes twonotebook computers 192 (e.g., Dell Precision M90s) networked using aCisco Gigabit Router.

The imaging software 38 is the Silicon Imaging SiliconDVR program withmulti-camera and stereo visualization, recording and control toolsrunning under Microsoft Windows. Other software tools used by theSiliconDVR for image processing include Microsoft Direct-X graphics,CineFormRAW Codec and Iridas Speedgrade 3D look-up-table software.

The software 38 is running on both notebook computers 192 and can beconfigured to view and control multiple camera modules 54 using a singledisplay with mixed, stitched or perspective-corrected views. The datafrom the multiple camera modules 54 can be recorded on either computer192 or both computers 192 simultaneously with multicasting enabled.

As seen in FIG. 10, another embodiment of the present inventionillustrates a digital camera system 200, similar to the digital camerasystems 50, 80, 100, 120, 130, 140, 150 and 180 described above, thedisclosed Mobile Stereo camera and recording system. It comprisingstereo pairs of camera modules, as described in FIGS. 1, 2, 3 and 4 onstereo rigs connected to a Frame grabber processing system as describedabove in FIG. 8.

FIG. 11 illustrates an alternate embodiment of the present inventionwhere a digital camera system 300, similar to the digital camera systems50, 80, 100, 120, 130, 140, 150, 180 and 200 described above, comprisinga plurality of multiple camera modules, stereo rigs, mobile recordersand remote clients operating any of the subsystems via wired or wirelessnetwork, producing live 2D or 3D content for broadcast or networkstreaming.

FIG. 12 illustrates a flow chart 400 for the software 38 describedabove. The software 38 searches for cameras, including sensor units andcamera modules, connected to the acquisition system via a data link,such as network, cameralink and HD-SDI with reverse serial communicationand control channel. Once detected, the software identifies the cameras402 by reading and determines make, model and unique serial numberidentifier, as well as any metadata stored in the camera or from deviceswhich may be connected, such as sensors, optics, lens control and motioncontrol systems. The software uses then adjusts settings and calibration404 the unique identifier to load previously generated and storedcalibration data for use in the live acquisition video processingsoftware pipeline 406. If the data does not exist, the software caninitiate a calibration process to generate new calibration data. Thesoftware receives input settings 408 thru the camera hardware or fromthe software generated user interface, which may be touchscreen controlsfor adjusting functions such as camera settings, recorder settings,image processing modes, streaming and playback. The softwarecontinuously captures images and metadata 410, including audio, externaldevice settings and operator settings. The image is then corrected,using the calibration parameters, including pixel-by-pixel black levelcorrection and defect pixel replacement then packed 412 and placed intoa RAM buffer 414 for additional processing. In Live Preview 416, variousadjustments represented in the metadata, can be applied to the motionimagery 418 such as white balance, color correction, gamma and threedimensional look-up-tables, such as those generated by tools such asIridas Speedgrade. Then additional visualization and image processing420 of the data can be applied such as scaling, zooming, exposuremetering, focus enhancing, flipping, keying, guide overlays and stereomixing. The imagery can then be output to displays. One display may beused for live motion image and operator touchscreen interface 422.Another display can be used for full screen live motion image outputwith or without overlay or data mixing and alternative visualization andprocessing 424. In systems with multiple image sources imagery from eachsource may be output to its own output or mixed for stereo 3Dvisualization, including driving outputs for mesh projection systems.Display settings such as convergence of dual camera streams can be sentinto the recording engine for storage with other metada. While the livemotion image is captured and processed, the software can record andstream the images and metadata 426. Image and metadata captured in theRAM buffer 414 can be accessed for optional encoding, packing andbuffering 428 in a first-in-first out basis at a different rate and froma different position of RAM buffer 414 than being accessed by the livepreview software engine. The encoding may use Cineform wavelet-basedcompression engine. The coded or uncoded data can then be written tofile in a container format such as Audio Video Interleaved .AVI,Quicktime .MOV or sequence of .DNG or raw images with an extension suchas .SIV 430. The file can be written to local or remote storage 432 on aproject and auto file sequence naming basis. The file can also bestreamed 438 via network 440. For continuous recording, the storage canspan multiple storage devices with programmable file size segmentation.The capture or recorded image and data can be retrieved from storage andplaced back into the RAM buffer 414 for playback. The playback enginecan access the images and metadata from the ram buffer 414 where it canbe decoded and unpacked and processed thru the same processing, displayvisualization and image processing as done during the live process, fordisplay on one or more displays. The playback data may also be combinedusing image processing and visualization with live image preview imagesto perform functions such as keying of live action with previouslyrecorded content. The live and playback processing can be done witheither single source or multiple sources including stereo playbackmixing with live stereo preview. The software reads data 438 fromstorage 440. The imagery for playback or live preview may also come foranother source, streamed 434 via network 436 into the RAM buffer. Therecording and streaming engines can also be used for processing of theimage processing visualized output content. Although the presentinvention has been discussed above in connection with use for film andvideo motion pictures, the present invention is not limited to thatenvironment and may also be used in any environment where digitalcameras may be employed including, without limitation, a security camerasystem, a digital camera system on a robotic vehicle or the like.

Although the present invention has been discussed above in connectionwith use for film and video motion pictures, the present invention isnot limited to that environment and may also be used in any environmentwhere digital cameras may be employed including, without limitation, asecurity camera system, a digital camera system on a robotic vehicle orthe like.

Although several embodiments have been described in detail for purposesof illustration, various modifications may be made without departingfrom the scope and spirit of the invention.

1. A high definition digital camera system comprising: an opticalassembly for gathering light from a desired scene; and an imaging systemcomprising a imager including an array of pixels aligned with theoptical assembly, and a minimum resolution and frame rate to produce aserial data rate of at least 48 megabytes per second, wherein the arrayproduces a minimum of 10-bits per pixel of color data; and a framegrabber for capturing raw image data output from the imager on aselective continuous or intermittent basis; wherein light gathered bythe optical assembly is received by the imager for generating andoutputting raw image data at film or video rates.
 2. The digital camerasystem of claim 1, wherein a lens mount interconnects the opticalassembly and modular imaging system, provides for film or video lensesto be removably connected to the modular imaging system.
 3. The digitalcamera system of claim 1, further comprising a processing systemreceiving output from the frame grabber, providing image processing andcommunication of the output from the frame grabber, wherein theprocessing unit executes reprogrammable software to perform an imageprocessing function for visualization, analysis, or storage.
 4. Thedigital camera system of claim 3, wherein the image processing functionis selected from one or more of a group consisting of image correction,defect pixel detection or correction, interpolation, white balance,white balance extraction, color correction, 3D look-up-table colortransformation, 3D look-up-table creation, pixel shading, motiondetection, audio interleaving, metadata interleaving, object detectionand classification, tracking, triangulation, calibration, color keying,image mixing, image saving, indicator overlay, focus detection andoverlay, exposure metering, histogram, false color data representation,zooming, scaling, data packing, pattern matching, pattern recognition,biometric recognition, data rate analysis, image enhancement,compression, decompression, transmission, recording or playback.
 5. Thedigital camera system of claim 3, wherein the optical assembly and theimaging system are located remotely from the processing unit.
 6. Thedigital camera system of claim 3, wherein the processing system performsprocessing, recording or display functions at a location remote to theimaging system.
 7. The digital camera system of claim 1, wherein thearray of pixels comprises a plurality of arrays of pixels.
 8. Thedigital camera system of claim 1, wherein the frame grabber outputsmotion picture imagery directly to a display.
 9. The digital camerasystem of claim 1, wherein the imaging system receives control signalsfor timing synchronization or record initiation.
 10. The digital camerasystem of claim 1, wherein a number of the pixels of the array from theimage sensor is selectively scanned and transmitted to the frame grabberand wherein the size and position of the subset is controlled by one ofthe frame grabber or processor.
 11. The digital camera system of claim1, wherein the imager is time synchronized with audio input.
 12. Thedigital camera system of claim 1, wherein the frame grabber can transmitnon-interpolated color filtered pixel data and uncompressed data. 13.The digital camera system of claim 1, wherein the data can beselectively be coded or not coded, and a wavelet codec having at least10-bits per pixel precision input codes the data.
 14. The digital camerasystem of claim 13, wherein the coded data is entirely stored on aremovable storage media with a minimum sustained writing speed of 10Megabytes per second.
 15. The digital camera system of claim 1, furthercomprising a communication interface providing for transmission of imagedata to a remote recording, processing or visualization system.
 16. Thedigital camera system of claim 1, wherein the frame grabber includes abuffer having capacity to capture images at rates faster than channelcapacity and transmits selective images or data.
 17. A method forrecording, editing and visualizing images taken by a digital cinemacamera system, comprising: generating scalable resolution, bit-depth andframe rate raw or color processed images from one or imaging modules atprecise film or video rates; capturing the raw or color processedimages; processing the raw or color processed images;
 18. The method ofclaim 17, further comprising providing a mechanism for timingsynchronization of exposure and readout cycles from the one or moreimaging modules.
 19. The method of claim 17, further comprisingcombining live imagery with previously stored imagery or computergenerated virtual sets while simultaneously recording the raw, broadcastformat, or visualization processed imagery in its original or nearoriginal representation.
 20. The method of claim 17, further comprisingusing a unified software or operator interface to control the capture,processing and non-destructive visualization from one or more imagingmodules.
 21. The method of claim 17, further comprising transmittingeither the raw sensor data or processed raw on the same or differentlinks utilizing industry standard cabling infrastructure.
 22. The methodof claim 17, further comprising generating multiple streams including afirst stream for recording at high data rates and at least oneadditional stream for recording at lower data rates for remotetransmission.
 23. The method of claim 17, further comprising recordingone or multiple image streams using a common removable storage device.24. The method of claim 17, further comprising mixing the data from themultiple data streams for outputting to a display.
 25. The method ofclaim 24, further comprising utilizing metadata encoded in a recordeddata stream to adjust stereographic effect and depth perception on thedisplay.